Retinal image capturing

ABSTRACT

An apparatus for producing a fundus image includes: a processor and a memory; an illumination component including a light source; a camera; and a display, the memory stores instructions that, when executed by the processor, cause the apparatus to: display a target element on the display; display a light source reflection from a cornea of an eye on the display; update a position of the light source reflection on the display as a caregiver manipulates a position of the apparatus relative to the eye; as time elapses, modify the display to allow the light source reflection to more easily be positioned within the target element; and automatically initiate fundus image capture with the camera when the light source reflection is within the target element on the display.

RELATED APPLICATION(S)

This patent application is related to U.S. patent application Ser. No.15/054,558 filed on Feb. 26, 2016, the entirety of which is herebyincorporated by reference.

INTRODUCTION

People with type 1 or type 2 diabetes can develop eye disease as aresult of having diabetes. One of the most common diabetic eye diseasesis diabetic retinopathy, which is damage to the blood vessels of thelight-sensitive tissue at the back of the eye, known as the retina.Trained medical professionals use cameras during eye examinations fordiabetic retinopathy screening. The cameras can produce images of theback of the eye and trained medical professionals use those images todiagnose and treat diabetic retinopathy.

These images are produced either with pharmacological pupil dilation,known as mydriatic fundus imaging, or without pharmacological pupildilation, known as non-mydriatic fundus imaging. Because pupil dilationis inversely related, in part, to the amount of ambient light,non-mydriatic fundus imaging usually occurs in low lightingenvironments. Medical professionals can also use fundus imagingapparatus to detect or monitor other diseases, such as hypertension,glaucoma, and papilledema.

SUMMARY

An example apparatus for producing a fundus image includes: a processorand a memory; an illumination component including a light source; acamera; and a display, the memory stores instructions that, whenexecuted by the processor, cause the apparatus to: display a targetelement on the display; display a light source reflection from a corneaof an eye on the display; update a position of the light sourcereflection on the display as a caregiver manipulates a position of theapparatus relative to the eye; as time elapses, modify the display toallow the light source reflection to more easily be positioned withinthe target element; and automatically initiate fundus image capture withthe camera when the light source reflection is within the target elementon the display.

DESCRIPTION OF THE FIGURES

The following figures, which form a part of this application, areillustrative of described technology and are not meant to limit thescope of the claims in any manner, which scope shall be based on theclaims appended hereto.

FIG. 1 is an embodiment of an example system for recording and viewingan image of a patient's fundus;

FIG. 2 is an embodiment of an example fundus imaging system;

FIG. 3 is an embodiment of an example method for imaging a patient'sfundus using a fundus imaging system;

FIG. 4 is an embodiment of an example fundus imaging system;

FIG. 5 illustrates an example method of initiating a fundus imagingusing passive eye tracking;

FIG. 6 is an embodiment of an example use of a fundus imaging system;

FIG. 7 is an example computing device used within the fundus imagingsystem;

FIG. 8 is another embodiment of an example fundus imaging system;

FIG. 9 is another view of the fundus imaging system of FIG. 8;

FIG. 10 is another view of the fundus imaging system of FIG. 8;

FIG. 11 is another view of the fundus imaging system of FIG. 8;

FIG. 12 is another view of the fundus imaging system of FIG. 8;

FIG. 13 is another view of the fundus imaging system of FIG. 8;

FIG. 14 is another view of the fundus imaging system of FIG. 8;

FIG. 15 is another view of the fundus imaging system of FIG. 8;

FIG. 16 is another embodiment of an example fundus imaging system;

FIG. 17 is another view of the fundus imaging system of FIG. 16;

FIG. 18 is another view of the fundus imaging system of FIG. 16;

FIG. 19 is another view of the fundus imaging system of FIG. 16;

FIG. 20 is another view of the fundus imaging system of FIG. 16;

FIG. 21 is another view of the fundus imaging system of FIG. 16;

FIG. 22 is another view of the fundus imaging system of FIG. 16;

FIG. 23 is another view of the fundus imaging system of FIG. 16;

FIG. 24 is another view of the fundus imaging system of FIG. 16;

FIG. 25 is another view of the fundus imaging system of FIG. 16;

FIG. 26 is another view of the fundus imaging system of FIG. 16;

FIGS. 27A and 27B are other views of the fundus imaging system of FIG.16 in use with a patient;

FIG. 28 is an embodiment of an example eye cup for use with the fundusimaging system of FIG. 8;

FIG. 29 is another view of the eye cup of FIG. 28;

FIG. 30 is another view of the eye cup of FIG. 28;

FIG. 31 is another view of the eye cup of FIG. 28;

FIG. 32 is another view of the eye cup of FIG. 28;

FIG. 33 is another embodiment of an example system for recording andviewing an image of a patient's fundus;

FIG. 34 is an example method for sending messages to an apparatus forrecording and viewing an image of a patient's fundus in the system ofFIG. 33;

FIG. 35 is an example message from the method of FIG. 34;

FIG. 36 is an example workflow for automatically capturing fundus imagesusing the system of FIG. 33;

FIG. 37 is an example graphical user interface that allows for images tobe added to the system of FIG. 33;

FIG. 38 is an example graphical user interface that allows for themanual capture of images using the system of FIG. 33;

FIG. 39 is an example graphical user interface that allows forpre-selection of an eye position and fixation target using the system ofFIG. 33;

FIG. 40 is an example graphical user interface to assist with aimingduring capture of images using the system of FIG. 33;

FIG. 41 is another example graphical user interface to assist withaiming during capture of images using the system of FIG. 33;

FIG. 42 is another view of the graphical user interface of FIG. 41 witha size of a target increased;

FIG. 43 is another view of the graphical user interface of FIG. 41 witha size of a target further increased;

FIG. 44 is another view of the graphical user interface of FIG. 41 witha size of a target further increased;

FIG. 45 is an example method for modifying a graphical user interface toassist with aiming during capture of images.

DETAILED DESCRIPTION

FIG. 1 is a schematic block diagram illustrating an example system 100for recording and viewing an image of a patient's fundus. In thisexample, the system 100 includes a patient P, a fundus imaging system102, a computing device 1800 including an image processor 106, a camera104 in communication with the computing device 1800, a display 108 incommunication with the computing device 1800 and used by clinician C,and a network 110. An embodiment of the example fundus imaging system102 is shown and described in more detail below with reference to FIG.4.

The fundus imaging system 102 functions to create a set of digitalimages of a patient's P eye fundus. As used herein, “fundus” refers tothe eye fundus and includes the retina, optic nerve, macula, vitreous,choroid and posterior pole.

In this example, one or more images of the eye are desired. Forinstance, the patient P is being screened for an eye disease, such asdiabetic retinopathy. The fundus imaging system 102 can also be used toprovide images of the eye for other purposes, such as to diagnose ormonitor the progression of a disease such as diabetic retinopathy.

The fundus imaging system 102 includes a handheld housing that supportsthe system's components. The housing supports one or two apertures forimaging one or two eyes at a time. In embodiments, the housing supportspositional guides for the patient P, such as an optional adjustable chinrest. The positional guide or guides help to align the patient's P eyeor eyes with the one or two apertures. In embodiments, the housingsupports means for raising and lowering the one or more apertures toalign them with the patient's P eye or eyes. Once the patient's P eyesare aligned, the clinician C then initiates the image captures by thefundus imaging system 102.

One technique for fundus imaging requires mydriasis, or the dilation ofthe patient's pupil, which can be painful and/or inconvenient to thepatient P. Example system 100 does not require a mydriatic drug to beadministered to the patient P before imaging, although the system 100can image the fundus if a mydriatic drug has been administered.

The system 100 can be used to assist the clinician C in screening for,monitoring, or diagnosing various eye diseases, such as hypertension,diabetic retinopathy, glaucoma and papilledema. It will be appreciatedthat the clinician C that operates the fundus imaging system 102 can bedifferent from the clinician C evaluating the resulting image.

In the example embodiment 100, the fundus imaging system 102 includes acamera 104 in communication with an image processor 106. In thisembodiment, the camera 104 is a digital camera including a lens, anaperture, and a sensor array. The camera 104 lens is a variable focuslens, such as a lens moved by a step motor, or a fluid lens, also knownas a liquid lens in the art. The camera 104 is configured to recordimages of the fundus one eye at a time. In other embodiments, the camera104 is configured to record an image of both eyes substantiallysimultaneously. In those embodiments, the fundus imaging system 102 caninclude two separate cameras, one for each eye.

In example system 100, the image processor 106 is operatively coupled tothe camera 104 and configured to communicate with the network 110 anddisplay 108.

The image processor 106 regulates the operation of the camera 104.Components of an example computing device, including an image processor,are shown in more detail in FIG. 7, which is described further below.

The display 108 is in communication with the image processor 106. In theexample embodiment, the housing supports the display 108. In otherembodiments, the display connects to the image processor, such as asmart phone, tablet computer, or external monitor. The display 108functions to reproduce the images produced by the fundus imaging system102 in a size and format readable by the clinician C. For example, thedisplay 108 can be a liquid crystal display (LCD) and active matrixorganic light emitting diode (AMOLED) display. The display can be touchsensitive.

The example fundus imaging system 102 is connected to a network 110. Thenetwork 110 may include any type of wireless network, a wired network,or any communication network known in the art. For example, wirelessconnections can include cellular network connections and connectionsmade using protocols such as 802.11a, b, and/or g. In other examples, awireless connection can be accomplished directly between the fundusimaging system 102 and an external display using one or more wired orwireless protocols, such as Bluetooth, Wi-Fi Direct, radio-frequencyidentification (RFID), or Zigbee. Other configurations are possible.

FIG. 2 illustrates components of an example fundus imaging system 102.The example fundus imaging system 102 includes a variable focus lens180, an illumination LED 182, an image sensor array 186, a fixation LED184, a computing device 1800, and a display 108. Each component is inelectrical communication with, at least, the computing device 1800.Other embodiments can include more or fewer components.

In one of the embodiments, the variable focus lens 180 is a liquid lens.A liquid lens is an optical lens whose focal length can be controlled bythe application of an external force, such as a voltage. The lensincludes a transparent fluid, such as water or water and oil, sealedwithin a cell and a transparent membrane. By applying a force to thefluid, the curvature of the fluid changes, thereby changing the focallength. This effect is known as electrowetting.

Generally, a liquid lens can focus between about −10 diopters to about+30 diopters. The focus of a liquid lens can be made quickly, even withlarge changes in focus. For instance, some liquid lenses can autofocusin tens of milliseconds or faster. Liquid lenses can focus from about 10cm to infinity and can have an effective focal length of about 16 mm orshorter.

In another embodiment of example fundus imaging system 102, the variablefocus lens 180 is one or more movable lenses that are controlled by astepping motor, a voice coil, an ultrasonic motor, or a piezoelectricactuator. Additionally, a stepping motor can also move the image sensorarray 186. In those embodiments, the variable focus lens 180 and/or theimage sensor array 186 are oriented normal to an optical axis of thefundus imaging system 102 and move along the optical axis. An examplestepping motor is shown and described below with reference to FIG. 4.

The example fundus imaging system 102 also includes an illuminationlight-emitting diode (LED) 182. The illumination LED 182 can be singlecolor or multi-color. For example, the illumination LED 182 can be athree-channel RGB LED, where each die is capable of independent andtandem operation. The illumination LED 182 can also include white LED(visible light LED) and Infrared (IR) LED. The visible light LED is forcapturing a color fundus image. The IR LED is for previewing the fundusduring the focusing and locating the fundus field of view whileminimizing disturbance of a patient's eye because the IR light source isnot visible by human's eye.

Optionally, the illumination LED 182 is an assembly including one ormore visible light LEDs and a near-infrared LED. The optionalnear-infrared LED can be used in a preview mode, for example, for theclinician C to determine or estimate the patient's P eye focus withoutilluminating visible light that could cause the pupil to contract orirritate the patient P.

The illumination LED 182 is in electrical communication with thecomputing device 1800. Thus, the illumination of illumination LED 182 iscoordinated with the adjustment of the variable focus lens 180 and imagecapture. The illumination LED 182 can be overdriven to draw more thanthe maximum standard current draw rating. In other embodiments, theillumination LED 182 can also include a near-infrared LED. Thenear-infrared LED is illuminated during a preview mode.

The example fundus imaging system 102 also optionally includes afixation LED 184. The fixation LED 184 is in communication with thecomputing device 1800 and produces a light to guide the patient's P eyefor alignment. The fixation LED 184 can be a single color or multicolorLED. For example, the fixation LED 184 can produce a beam of green lightthat appears as a green dot when the patient P looks into the fundusimaging system 102. Other colors and designs, such as a cross, “x” andcircle are possible.

The example fundus imaging system 102 also includes an image sensorarray 186 that receives and processes light reflected by the patient'sfundus. The image sensor array 186 is, for example, a complementarymetal-oxide semiconductor (CMOS) sensor array, also known as an activepixel sensor (APS), or a charge coupled device (CCD) sensor.

The image sensor array 186 has a plurality of rows of pixels and aplurality of columns of pixels. In some embodiments, the image sensorarray has about 1280 by 1024 pixels, about 640 by 480 pixels, about 1500by 1152 pixels, about 2048 by 1536 pixels, or about 2560 by 1920 pixels.

In some embodiments, the pixel size in the image sensor array 186 isfrom about four micrometers by about four micrometers; from about twomicrometers by about two micrometers; from about six micrometers byabout six micrometers; or from about one micrometer by about onemicrometer.

The example image sensor array 186 includes photodiodes that have alight-receiving surface and have substantially uniform length and width.During exposure, the photodiodes convert the incident light to a charge.The image sensor array 186 can be operated as a global reset, that is,substantially all of the photodiodes are exposed simultaneously and forsubstantially identical lengths of time.

The example fundus imaging system 102 also includes a display 108,discussed in more detail above with reference to FIG. 1. Additionally,the example fundus imaging system 102 includes a computing device 1800,discussed in more detail below with reference to FIG. 7.

FIG. 3 is an embodiment of a method 200 for imaging a patient's fundususing a fundus imaging system. In the embodiment shown, the lighting isoptimally dimmed prior to execution, although lowering the lighting isoptional. The embodiment shown includes a set depth of field operation204, a set number of zones operation 206, an illuminate lightingoperation 208, an adjust lens focus operation 210, a capture imageoperation 212, repeat operation(s) 213, a show images operation 214 anda determine representative image operation 216. Other embodiments caninclude more or fewer steps.

The embodiment of method 200 begins with setting a depth of fieldoperation 204. In embodiments, the variable focus lens 180 is capable offocusing from about −20 diopters to about +20 diopters. Set depth offield operation 204 defines the lower and upper bounds in terms ofdiopters. For example, the depth of field range could be set to about−10 to +10 diopters; about −5 to about +5 diopters; about −10 to about+20 diopters; about −5 to about +20 diopters; about −20 to about +0diopters; or about −5 to about +5 diopters. Other settings are possible.The depth of field can be preprogrammed by the manufacturer.Alternatively, the end user, such as the clinician C, can set the depthof field.

As shown in FIG. 3, the next operation in embodiment of method 200 issetting the number of zones operation 206. However, zones operation 206can occur before or concurrent with field operation 204. In zonesoperation 206, the depth of field is divided into equal parts, whereeach part is called a zone. In other embodiments, the zones are not allequal. The number of zones is equal to the number of images captured incapture image operation 212.

For example, when the depth of field is from −10 to +10 diopters, thefocus of the variable focus lens can be changed by 4 diopters beforeeach image capture. Thus, in this example, images would be captured at−10, −6, −2, +2, +6 and +10 diopters. Or, images could be captured at−8, −4, 0, +4 and +8 diopters, thereby capturing an image in zones −10to −6 diopters, −6 to −2 diopters, −2 to +2 diopters, +2 to +6 dioptersand +6 to +10 diopters, respectively. In that instance, the depth offocus is about +/−2 diopters. Of course, the number of zones and thedepth of field can vary, resulting in different ranges of depth of fieldimage capture.

In embodiments, both depth of field and number of zones arepredetermined. For example, −10 D to +10 D and 5 zones. Both can bechanged by a user.

After the depth of field and number of zones are set, the next operationin embodiment of method 200 is the image capture process, which includesilluminate lighting operation 208, adjust lens focus operation 210 andcapture image operation 212. As shown in FIG. 3, the lighting componentis illuminated (lighting operation 208) before the lens focus isadjusted (lens focus operation 210). However, lens focus operation 210can occur before or concurrent with lighting operation 208.

The illumination LED 182 is illuminated in lighting operation 208. Theillumination LED 182 can remain illuminated throughout the duration ofeach image capture. Alternatively, the illumination LED 182 can beturned on and off for each image capture. In embodiments, theillumination LED 182 only turns on for the same period of time as theimage sensor array 186 exposure time period.

Optionally, lighting operation 208 can additionally include illuminatinga near-infrared LED. The clinician C can use the illumination of thenear-infrared LED as a way to preview the position of the patient's Ppupil.

The focus of variable focus lens 180 is adjusted in lens focus operation210. Autofocusing is not used in embodiment of method 200. That is, thediopter setting is provided to the lens without regard to the quality ofthe focus of the image. Indeed, traditional autofocusing fails in thelow-lighting non-mydriatic image capturing environment. The embodimentof method 200 results in a plurality of images at least one of which, ora combination of which, yields an in-focus view of the patient's Pfundus.

Additionally, the lack of autofocusing enables the fundus imaging system102 to rapidly capture multiple images in capture image operation 212 atdifferent diopter ranges. That is, variable focus lens 180 can be set toa particular diopter range and an image captured without the systemverifying that the particular focus level will produce an in-focusimage, as is found in autofocusing systems. Because the system does notattempt to autofocus, and the focus of the variable focus lens 180 canbe altered in roughly tens of milliseconds, images can be capturedthroughout the depth of field in well under a second, in embodiments.Thus, in the embodiment of method 200, the fundus imaging system 102 cancapture images of the entire depth of field before the patient's P eyecan react to the illuminated light. Without being bound to a particulartheory, depending on the patient P, the eye might react to the lightfrom illumination of white LED 182 in about 150 milliseconds.

The image sensor array 186 captures an image of the fundus in captureimage operation 212. As discussed above, the embodiment of method 200includes multiple image captures of the same fundus at different diopterfoci. The example fundus imaging system 102 uses a global reset orglobal shutter array, although other types of shutter arrays, such as arolling shutter, can be used. The entire image capture method 200 canalso be triggered by passive eye tracking and automatically capture, forexample, 5 frames of images. An embodiment of example method for passiveeye tracking is shown and described in more detail with reference toFIG. 5, below.

After the fundus imaging system 102 captures an image of the fundus, theembodiment of method 200 returns in loop 213 to either the illuminatelighting operation 208 or the adjust lens focus operation 210. That is,operations 208, 210 and 212 are repeated until an image is captured ineach of the preset zones from zones operation 206. It is noted that theimage capture does not need to be sequential through the depth of field.Additionally, each of the images does not need to be captured in asingle loop; a patient could have one or more fundus images captured andthen one or more after a pause or break.

After an image is captured in each of the zones (capture image operation212) in embodiment of method 200, either the images are displayed inshow images operation 214 or a representative image is determined inoperation 216 and then the image is displayed. Show images operation 214can include showing all images simultaneously or sequentially on display108. A user interface shown on display 108 can then enable the clinicianC or other reviewing medical professional to select or identify the bestor a representative image of the patient's P fundus.

In addition to, or in place of, show images operation 214, the computingdevice can determine a representative fundus image in operation 216.Operation 216 can also produce a single image by compiling aspects ofone or more of the images captured. This can be accomplished by, forexample, using a wavelet feature reconstruction method to select,interpolate, and/or synthesize the most representative frequency orlocation components.

The fundus imaging system 102 can also produce a three-dimensional imageof the fundus by compiling the multiple captured images. Because theimages are taken at different focus ranges of the fundus, thecompilation of the pictures can contain three-dimensional informationabout the fundus.

In turn, the image or images from operation 214 or 216 can be sent to apatient's electronic medical record or to a different medicalprofessional via network 110.

FIG. 4 illustrates an embodiment of example fundus imaging system 400.The embodiment 400 includes a housing 401 that supports an optionalfixation LED 402, an objective lens 404, fixation LED mirrors 405,variable focus lens assembly 406, display 408, printed circuit board410, step motor 412, image sensor array 414, and illumination LED 416.Also shown in FIG. 4 are light paths L that include potential lightpaths from optional fixation LED 402 and incoming light paths fromoutside the fundus imaging system 400. The illustrated components havethe same or similar functionality to the corresponding componentsdiscussed above with reference to FIGS. 1-3 above. Other embodiments caninclude more or fewer components.

The housing 401 of example fundus imaging system 400 is sized to be handheld. In embodiments, the housing 401 additionally supports one or moreuser input buttons near display 408, not shown in FIG. 4. The user inputbutton can initiate the image capture sequence, at least a portion ofwhich is shown and discussed with reference to FIG. 3, above. Thus, thefundus imaging system 400 is capable of being configured such that theclinician C does not need to adjust the lens focus.

Fixation LED 402 is an optional component of the fundus imaging system400. The fixation LED 402 is a single or multi-colored LED. Fixation LED402 can be more than one LED.

As shown in FIG. 4, pivoting mirrors 405 can be used to direct lightfrom the fixation LED 402 towards the patient's pupil. Additionally, anoverlay or filter can be used to project a particular shape or image,such as an “X”, to direct the patient's focus. The pivoting mirrors 405can control where the fixation image appears in the patient's view. Thepivoting mirrors 405 do not affect the light reflected from thepatient's fundus.

The embodiment of example fundus imaging system 400 also includes avariable focus lens assembly 406. As shown in FIG. 4, the variable focuslens assembly 406 is substantially aligned with the longitudinal axis ofthe housing 401. Additionally, the variable focus lens assembly 406 ispositioned between the objective lens 404 and the image sensor array 414such that it can control the focus of the incident light L onto theimage sensor array.

The example printed circuit board 410 is shown positioned within onedistal end of the housing 401 near the display 408. However, the printedcircuit board 410 can be positioned in a different location. The printedcircuit board 410 supports the components of the example computingdevice 1800. A power supply can also be positioned near printed circuitboard 410 and configured to power the components of the embodiment ofexample fundus imaging system 400.

Step motor 412 is an optional component in the example embodiment 400.Step motor 412 can also be, for example, a voice coil, an ultrasonicmotor, or a piezoelectric actuator. In the example embodiment 400, stepmotor 412 moves the variable focus lens assembly 406 and/or the sensorarray 414 to achieve variable focus. The step motor 412 moves thevariable focus lens assembly 406 or the sensor array 414 in a directionparallel to a longitudinal axis of the housing 401 (the optical axis).The movement of step motor 412 is actuated by computing device 1800.

The example image sensor array 414 is positioned normal to thelongitudinal axis of the housing 401. As discussed above, the imagesensor array 414 is in electrical communication with the computingdevice. Also, as discussed above, the image sensor array can be a CMOS(APS) or CCD sensor.

An illumination LED 416 is positioned near the variable focus lensassembly 406. However, the illumination LED 416 can be positioned inother locations, such as near or with the fixation LED 402.

FIG. 5 illustrates an alternate embodiment of initiate retinal imagingstep 306 using passive eye tracking. The initiate retinal imaging step306 operates to image the fundus of the patient P using passive eyetracking. In the initiate retinal imaging step 306, the fundus imagingsystem 102 monitors the pupil/fovea orientation of the patient P.Although the initiate retinal imaging step 306 is described with respectto fundus imaging system 102, the initiate retinal imaging step 306 maybe performed using a wearable or nonwearable fundus imaging system, suchas a handheld digital fundus imaging system.

Initially, at step 303, the pupil or fovea or both of the patient P aremonitored. The fundus imaging system 102 captures images in a firstimage capture mode. In the first image capture mode, the fundus imagingsystem 102 captures images at a higher frame rate. In some embodiments,in the first image capture mode, the fundus imaging system 102 capturesimages with infra-red illumination and at lower resolutions. In someembodiments, the infra-red illumination is created by the illuminationLED 182 operating to generate and direct light of a lower intensitytowards the subject. The first image capture mode may minimizediscomfort to the patient P, allow the patient P to relax, and allow fora larger pupil size without dilation (non-mydriatic).

Next, at step 305, the computing device 1800 processes at least aportion of the images captured by the fundus imaging system 102. Thecomputing device 1800 processes the images to identify the location ofthe pupil or fovea or both of the patient P. Using the location of thepupil or fovea or both in one of the images, a vector corresponding tothe pupil/fovea orientation is calculated. In some embodiments, thepupil/fovea orientation is approximated based on the distance betweenthe pupil and fovea in the image. In other embodiments, the pupil/foveaorientation is calculated by approximating the position of the fovearelative to the pupil in three dimensions using estimates of thedistance to the pupil and the distance between the pupil and the fovea.In other embodiments, the pupil/fovea orientation is approximated fromthe position of the pupil alone. In yet other embodiments, other methodsof approximating the pupil/fovea orientation are used.

Next, at step 307, the pupil/fovea orientation is compared to theoptical axis of the fundus imaging system 102. If the pupil/foveaorientation is substantially aligned with the optical axis of the fundusimaging system 102, the process proceeds to step 309 to capture a fundusimage. If not, the process returns to step 303 to continue to monitorthe pupil or fovea. In some embodiments, the pupil/fovea orientation issubstantially aligned with the optical axis when the angle between themis less than two to fifteen degrees.

Next, at step 309, fundus images are captured by triggering theembodiment of example thru focusing image capturing method 200. Inembodiments, five images are captured at step 309. In some embodiments,the fundus image is captured in a second image capture mode. In someembodiments, in the second image capture mode, the fundus imaging system102 captures images with visible (or white) illumination and at higherresolutions. In some embodiments, the visible illumination is created bythe illumination LED 182 operating to generate and direct light of ahigher intensity towards the subject. In other embodiments, the higherillumination is created by an external light source or ambient light.The second image capture mode may facilitate capturing a clear,well-illuminated, and detailed fundus image.

In some embodiments, after step 309, the initiate retinal imaging step306 returns to step 303 to continue to monitor the pupil/foveaorientation. The initiate retinal imaging step 306 may continue tocollect fundus images indefinitely or until a specified number of imageshave been collected. Further information regarding passive eye trackingcan be found in U.S. patent application Ser. No. 14/177,594 filed onFeb. 11, 2014, titled Ophthalmoscope Device, which is herebyincorporated by reference in its entirety

FIG. 6 is an embodiment of example use 500 of fundus imaging system 102.In the embodiment of example use 500, a clinician positions the fundusimaging system (operation 502), initiates image capture (operation 504),positions the fundus imaging system over the other eye (operation 506),initiates image capture (operation 508), and views images (operation520). Although the example use 500 is conducted without firstadministering mydriatic pharmaceuticals, the example use 500 can also beperformed for a patient who has taken a pupil-dilating compound. Theembodiment of example use 500 can also include lowering the lighting.The embodiment of example use 500 is conducted using the same or similarcomponents as those described above with reference to FIGS. 1-3. Otherembodiments can include more or fewer operations.

The embodiment of example use 500 begins by positioning the fundusimaging system (operation 502). In embodiments, the clinician firstinitiates an image capture sequence via a button on the housing or agraphical user interface shown by the display. The graphical userinterface can instruct the clinician to position the fundus imagingsystem over a particular eye of the patient. Alternatively, theclinician can use the graphical user interface to indicate which eyefundus is being imaged first.

In operation 502, the clinician positions the fundus imaging system nearthe patient's eye socket. The clinician positions the aperture of thesystem flush against the patient's eye socket such that the aperture, ora soft material eye cup extending from the aperture, seals out most ofthe ambient light. Of course, the example use 500 does not requirepositioning the aperture flush against the patient's eye socket.

When the fundus imaging system is in position, the system captures morethan one image of the fundus in operation 504. As discussed above, thesystem does not require the clinician to manually focus the lens.Additionally, the system does not attempt to autofocus on the fundus.Rather, the clinician simply initiates the image capture, via a buttonor the GUI, and the fundus imaging system controls when to capture theimages and the focus of the variable focus lens. Also, as discussedabove at least with reference to FIG. 5, the system can initiate imagecapture using passive eye tracking.

The patient may require the fundus imaging system to be moved away fromthe eye socket during image capture operation 504. The clinician canre-initiate the image capture sequence of the same eye using the buttonor the GUI on the display.

After capturing an image in each of the specified zones, the fundusimaging system notifies the clinician that the housing should bepositioned over the other eye (operation 506). The notification can beaudible, such as a beep, and/or the display can show a notification. Inembodiments, the system is configured to capture a set of images of onlyone eye, wherein the example method 500 proceeds to view imagesoperation 520 after image capture operation 504.

Similar to operation 502, the clinician then positions the fundusimaging system near or flush with the patient's other eye socket inoperation 506. Again, when the system is in place, an image is capturedin every zone in operation 508.

After images have been captured of the fundus in each pre-set zone, theclinician can view the resulting images in operation 520. As noted abovewith reference to FIG. 3, the images can be post-processed before theclinician views the images to select or synthesize a representativeimage. Additionally, the fundus images can be sent to a remote locationfor viewing by a different medical professional.

FIG. 7 is a block diagram illustrating physical components (i.e.,hardware) of a computing device 1800 with which embodiments of thedisclosure may be practiced. The computing device components describedbelow may be suitable to act as the computing devices described above,such as wireless computing device and/or medical device of FIG. 1. In abasic configuration, the computing device 1800 may include at least oneprocessing unit 1802 and a system memory 1804. Depending on theconfiguration and type of computing device, the system memory 1804 maycomprise, but is not limited to, volatile storage (e.g., random accessmemory), non-volatile storage (e.g., read-only memory), flash memory, orany combination of such memories. The system memory 1804 may include anoperating system 1805 and one or more program modules 1806 suitable forrunning software applications 1820. The operating system 1805, forexample, may be suitable for controlling the operation of the computingdevice 1800. Furthermore, embodiments of the disclosure may be practicedin conjunction with a graphics library, other operating systems, or anyother application program and is not limited to any particularapplication or system. This basic configuration is illustrated in FIG. 7by those components within a dashed line 1808. The computing device 1800may have additional features or functionality. For example, thecomputing device 1800 may also include additional data storage devices(removable and/or non-removable) such as, for example, magnetic disks,optical disks, or tape. Such additional storage is illustrated in FIG. 7by a removable storage device 1809 and a non-removable storage device1810.

As stated above, a number of program modules and data files may bestored in the system memory 1804. While executing on the at least oneprocessing unit 1802, the program modules 1806 may perform processesincluding, but not limited to, generate list of devices, broadcastuser-friendly name, broadcast transmitter power, determine proximity ofwireless computing device, connect with wireless computing device,transfer vital sign data to a patient's EMR, sort list of wirelesscomputing devices within range, and other processes described withreference to the figures as described herein. Other program modules thatmay be used in accordance with embodiments of the present disclosure,and in particular to generate screen content, may include electronicmail and contacts applications, word processing applications,spreadsheet applications, database applications, slide presentationapplications, drawing or computer-aided application programs, etc.

Furthermore, embodiments of the disclosure may be practiced in anelectrical circuit comprising discrete electronic elements, packaged orintegrated electronic chips containing logic gates, a circuit utilizinga microprocessor, or on a single chip containing electronic elements ormicroprocessors. For example, embodiments of the disclosure may bepracticed via a system-on-a-chip (SOC) where each or many of thecomponents illustrated in FIG. 7 may be integrated onto a singleintegrated circuit. Such an SOC device may include one or moreprocessing units, graphics units, communications units, systemvirtualization units and various application functionality all of whichare integrated (or “burned”) onto the chip substrate as a singleintegrated circuit. When operating via an SOC, the functionality,described herein, may be operated via application-specific logicintegrated with other components of the computing device 1800 on thesingle integrated circuit (chip). Embodiments of the disclosure may alsobe practiced using other technologies capable of performing logicaloperations such as, for example, AND, OR, and NOT, including but notlimited to mechanical, optical, fluidic, and quantum technologies. Inaddition, embodiments of the disclosure may be practiced within ageneral purpose computer or in any other circuits or systems.

The computing device 1800 may also have one or more input device(s) 1812such as a keyboard, a mouse, a pen, a sound or voice input device, atouch or swipe input device, etc. The output device(s) 1814 such as adisplay, speakers, a printer, etc. may also be included. Theaforementioned devices are examples and others may be used. Thecomputing device 1800 may include one or more communication connections1816 allowing communications with other computing devices. Examples ofsuitable communication connections 1816 include, but are not limited to,RF transmitter, receiver, and/or transceiver circuitry; universal serialbus (USB), parallel, and/or serial ports.

The term computer readable media as used herein may includenon-transitory computer storage media. Computer storage media mayinclude volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information, suchas computer readable instructions, data structures, or program modules.The system memory 1804, the removable storage device 1809, and thenon-removable storage device 1810 are all computer storage mediaexamples (i.e., memory storage.) Computer storage media may include RAM,ROM, electrically erasable read-only memory (EEPROM), flash memory orother memory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other article ofmanufacture which can be used to store information and which can beaccessed by the computing device 1800. Any such computer storage mediamay be part of the computing device 1800. Computer storage media doesnot include a carrier wave or other propagated or modulated data signal.

Communication media may be embodied by computer readable instructions,data structures, program modules, or other data in a modulated datasignal, such as a carrier wave or other transport mechanism, andincludes any information delivery media. The term “modulated datasignal” may describe a signal that has one or more characteristics setor changed in such a manner as to encode information in the signal. Byway of example, and not limitation, communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), infrared, andother wireless media.

Although the example medical devices described herein are devices usedto monitor patients, other types of medical devices can also be used.For example, the different components of the CONNEX™ system, such as theintermediary servers that communication with the monitoring devices, canalso require maintenance in the form of firmware and software updates.These intermediary servers can be managed by the systems and methodsdescribed herein to update the maintenance requirements of the servers.

Embodiments of the present invention may be utilized in variousdistributed computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network in adistributed computing environment.

The block diagrams depicted herein are just examples. There may be manyvariations to these diagrams described therein without departing fromthe spirit of the disclosure. For instance, components may be added,deleted or modified.

While embodiments have been described, it will be understood that thoseskilled in the art, both now and in the future, may make variousimprovements and enhancements can be made.

As used herein, “about” refers to a degree of deviation based onexperimental error typical for the particular property identified. Thelatitude provided the term “about” will depend on the specific contextand particular property and can be readily discerned by those skilled inthe art. The term “about” is not intended to either expand or limit thedegree of equivalents which may otherwise be afforded a particularvalue. Further, unless otherwise stated, the term “about” shallexpressly include “exactly,” consistent with the discussions regardingranges and numerical data. Concentrations, amounts, and other numericaldata may be expressed or presented herein in a range format. It is to beunderstood that such a range format is used merely for convenience andbrevity and thus should be interpreted flexibly to include not only thenumerical values explicitly recited as the limits of the range, but alsoto include all the individual numerical values or sub-ranges encompassedwithin that range as if each numerical value and sub-range is explicitlyrecited. As an illustration, a numerical range of “about 4 percent toabout 7 percent” should be interpreted to include not only theexplicitly recited values of about 4 percent to about 7 percent, butalso include individual values and sub-ranges within the indicatedrange. Thus, included in this numerical range are individual values suchas 4.5, 5.25 and 6 and sub-ranges such as from 4-5, from 5-7, and from5.5-6.5; etc. This same principle applies to ranges reciting only onenumerical value. Furthermore, such an interpretation should applyregardless of the breadth of the range or the characteristics beingdescribed.

Referring now to FIGS. 8-15, another example fundus imaging system 600is shown. The embodiment 600 is similar to the fundus imaging system 400described above.

The fundus imaging system 600 includes a housing 601 that supports adisplay 602 at a first end and an opposite end 603 configured to engagean eye of the patient. As described herein, the fundus imaging system600 can be used to implement one or more of the described methods forimaging of the fundus.

Yet another embodiment of an example fundus imaging system 605 is shownin FIGS. 16-26. In this example, the body of the fundus imaging system605 can be formed of two or more materials overmolded upon one another.For example, a first polymeric material can be used to form the mainbody, and a second, softer polymeric material can be overmolded onto thefirst material to form bumper and/or grip areas, as depicted in FIG. 26.These overmolded areas provide a softer and slip-resistant surface foreasier grapping and holding of the fundus imaging system 605. Themultiple gripping surfaces allow the clinician C to decide how best tohold the fundus imaging system 605 in use.

Referring now to FIGS. 27A and 27B, the fundus imaging system 605 isshown in use on the patient. The fundus imaging system 605 is placedwith an end (e.g., opposite end 603) adjacent to or touching thepatient's face surrounding the desired eye socket.

Specifically, an end 607 of an example eye cup 606, shown in FIGS.28-32, is positioned at the end 603 of the fundus imaging system 600 or605. An opposite end 608 is positioned again the eye socket surroundingthe eye for which imaging will occur. In this example, the eye cup 606is formed of a polymeric material that is flexible in an accordion-likemanner. This allows the fundus imaging system 600 or 605 to be moved bythe clinician C towards and away from the patient's eye while stillmaintaining contact with the patient's face. Other configurations arepossible.

In another example system 700 for recording and viewing an image of apatient's fundus shown in FIG. 33, the system 700 is cloud-based (e.g.,includes a plurality of servers with storage accessible from a largenetwork such as the Internet) and allows for communication and storageof fundus images across LANs, WANs, and the Internet. In this example, adevice 702, which is identical and/or similar to the systems 600, 605described above, can be used to capture an image, associate that imagewith a patient, review the image, and annotate the image as desired.

Upon completion, the image can be uploaded to a cloud system 704 using abatch or more instant configuration. When uploaded, the image can betagged with device and patient information, such as a barcode associatedwith the patient and/or a patient picture. The cloud system 704 can beconfigured to provide patient lists and to accept or reject an imagebased upon given criteria, such a patient name and quality of image. Thecloud system 704 can also be used to provide notifications, such asimage availability, to the clinician C and/or patient. In addition, thecloud can forward the image and patient information to an EMR 706 forstorage.

In addition, the cloud system 704 can be used to provide a portal toallow for access to images by a device 708 of the clinician C and/orpatient device 710 using a computing device such as a personal computingdevice, tablet, and/or mobile device. This can allow the images to beviewed, manipulated, etc. The cloud system 704 can be used to captureclinician C annotations and diagnoses. In addition, the cloud system 704can be configured to interface with other third parties, such asinsurance companies to allow for billing.

In some examples the systems 600, 605 can be configured to operate inboth manual and automatic modes when interfacing with the cloud system704. In one example, the automatic mode includes one or more scriptsthat automate certain processes for the systems 600, 605. See FIG. 36described below. These processes can include automation of image focusand capture (acquisition) and output to the cloud for storage. In themanual mode, the various processes can be manually controlled by theclinician C, such as focus on the fundus, capture of one or more imagesat desired times, and then uploading of the image(s) to the cloud. SeeFIG. 37 described below.

A notification scheme is used for charging of the systems 600, 605. Inthese examples, the systems 600, 605 are wireless and include arechargeable battery pack, such as a lithium-ion battery or similarbattery. In this example, a bi-color LED is used to indicate a status ofcharging of the battery pack when placed in a charging cradle 703. TheLED is left off if charging is not occurring—this is the default state.When the systems 600, 605 are charging (e.g., when plugged into a dock),the LED is illuminated a solid amber to indicate charging of the batteryand a solid green when the battery charging is completed. If an erroroccurs during charging, the LED flashes an amber color. Otherconfigurations are possible.

Different example operating states for the fundus imaging systems 600,605 are possible. For a clinician that gathers the images from thepatient, the systems 600, 605 can be used to select a patient, adjustthe eye cap, take an image, determine a quality of the image, and reviewthe status of an image capture process. In addition, various otherfeatures, such as adjustment of connectivity (e.g., WiFi) and cleaningof the device can be accomplished. Additional details on some of theseprocesses are provided below.

Further, a physician (sometimes the same individual who captured theimages or a different individual, such as an individual located at aremote location) can review the results of the image captures anddevelop/review a report based upon the same.

Example processes are performed in the cloud system 704 based upon eachindividual or service within the system. For the clinician capturing theimages, the cloud system 704 can be used to add new patients, schedulethe procedure, and check the status of the procedure. For the physicianreviewing the images, the cloud system 704 can be used to check status,review the images, and generate/review a report based upon review of theimages. Notifications can also be created and sent to, for example, theclinician or patient.

The systems 600, 605 can be used to transmit scheduling and/or imageinformation to and from the cloud system 704. The EMR 706 is incommunication with the cloud system 704 to transmit and store image anddiagnoses information for each patient. Other configurations arepossible.

An over read service 712 is also shown in FIG. 33. The over read service712 can interact with the cloud system 704 to provide additionalresources for analyzing the images, including reading of images andgenerating of reports. Other functions of the example system 700 includecapture and forwarding of images to the cloud and communication betweenthe cloud and the EMR 706 for storage thereof.

For example, in one embodiment, the device 702 is used to capture one ormore fundus images. After capture, the device 702 is returned to thecharging cradle 703. Upon placement of the device 702 into the cradle703, the captured images are automatically transferred to the cloudsystem 704. This transfer can be automated, so that no further action isrequired by the user to transfer the images from the device 702 to thecloud system 704.

Upon submission to the cloud system 704, the images can be automaticallyreviewed for quality. The images can also be automatically forwarded tothe over read service 712 for review. One or more clinicians canthereupon review the images and provide feedback from the over readservice 712 back to the cloud system 704. At this point, the cloudsystem 704 can provide notification to the devices 708, 710 regardingthe information from the over read service 712.

An example method for using the systems 600, 605 to capture fundusimages includes preliminary tasks such as the capturing of patientvitals and education of the patient on the procedure are done. Once thisis done, the system 600, 605 is powered on and the patient is selectedon the device. The eye cup is then positioned on the patient and one ormore images are captured using automated and/or manual processes. Theimages can then be checked. If accepted, the images are saved and/oruploaded to the cloud. The system 600, 605 can be powered off andreturned to its cradle for charging. A physician can thereupon reviewthe images, and the clinician C or patient can be notified of theresults.

In an example method for obtaining a good quality image of the fundususing the systems 600, 605, after an image is captured, the cliniciancan accept or reject the image. If rejected, a script can be executedthat provides manual or automated instructions on how to capture adesired image quality. The clinician thereupon gets another opportunityto capture an image and then to accept or reject it. If the image isaccepted, automated processes can be used to determine a quality of theimage. If accepted, further scripting can occur. If not, the cliniciancan be prompted to take another image.

An example method is provided to allow for capture of images even whenthe system 600, 605 loses connectivity with the cloud. In such aninstance, automated quality checks may not be provided, and theclinician may be prompted as such. The clinician can then decide whetheror not to accept the image without the quality check or to cancel theprocedure. In addition, the system 600, 605 can be used to trouble shootconnectivity issues, as described further below.

An example method for allowing the clinician to select the patient onthe system 600, 605 includes a work list that is provided thatidentifies patients based upon one or more given criteria, such as theclinician, location, time of day, etc. The clinician is thereupon ableto select the patient and confirms the proper patient has been selected,such as by comparing a code with one worn by the patient for from apicture of the patient. Thereupon, after selection of the patient, oneor more images can be captured and stored. The captured images areassociated with the selected patient.

In a similar manner, an example method allows the clinician to assurethat the proper patient is selected. Upon power-up of the system 600,605, unique information is sent to the cloud, such as the system'sserial number. The could looks-up the serial number and returns a listof patients associated with that system. The clinician can thereuponselect the patient from the list or manually enter the patient into thesystem 600, 605 if the patient is not on the work list.

A user interface allows the user to pick between a selection ofpatients, examinations, review, and settings. If a patient is selected,the system 600, 605 proceeds with imaging of the fundus using anautomated and/or manual process. Icons are used to represent differentcontexts on the user interfaces of the system 600, 605.

The following example workflows/methods are implemented by the systems600, 605. Additional details regarding these workflows can also be foundwith reference to FIGS. 36-44.

An example method for automatic examination and image capture startswhen the clinician selects the examination icon on the system 600, 605.Upon initiation, the clinician is presented with an interface thatallows for automatic acquisition of the fundus image. This can beaccomplished in three stages, including pre acquisition, acquisition,and post-acquisition. During pre-acquisition, the clinician selects thepatient and configures the system as desired. During acquisition, theimage is captured using automated or manual processes. Finally,post-acquisition, quality checks are performed and the clinician cansave the image(s) if desired. See FIG. 36 described further below.

An example method for adjusting certain settings of the system 600, 605includes, for example, brightness and focus, which can be selectedautomatically or manually manipulated by the clinician.

An example method for manually acquiring an image is similar to themethod described above, except the acquisition of the images is donemanually by the clinician. This is accomplished by the clinicianmanually indicating when an image is to be taken. Upon capture, theimage can be verified manually or automatically.

An example method for navigating one or more captured images includes auser interface that is used to scroll through the captured images in asequence. Upon review, the images can be submitted, if desired.

An example method for selecting a patient from a worklist starts uponselection of the patient icon from the interface for the system 600,605. A list of patients is presented to the clinician in the worklist.The clinician can select a patient from the list to be presented withadditional information about that patient, such as full name, date ofbirth, and patient ID. If any unsaved images exist, those images areassociated with the selected patient. If not, a new examination routineis executed to allow for capture of images to be associated with theselected patient.

An example method allows for the clinician to manually enter new patientinformation into the system 600, 605. This includes patient name, dateof birth, and/or patient ID. Once entered, the patient information canbe associated with captured images.

An example method allows the clinician to search for a specific patientusing such parameters as patient name, date of birth, and/or patient ID.Once found, the clinician selects the patient for further processing.

An example method for refreshing the patient worklist includes assumingthere is connectivity (e.g., to the cloud), the clinician selecting arefresh button to manually refresh the list with the most currentpatient names. The system 600, 605 is also programmed to periodicallyrefresh the list automatically at given intervals and at other givenperiods, such as upon startup or shutdown. Other configurations arepossible.

An example method allows a clinician to review a patient test on thesystem 600, 605. Upon selection of a patient, the clinician can reviewpatient summary information (e.g., full name, date of birth, and patientID) and previous examination summary information, such as items from theexamination and image quality scores, which indicate how good the imagequality was from those examinations.

An example method for saving images allows, after acquisition, theclinician to review the images in sequence. For each image in theworkspace, the image is quality-checked and the status of the image isdisplayed to the clinician. The clinician uses the user interface toreview each acquired image and to save or discard the image.

An example method labeling eye position allows the clinician to selectupon five eye positions, including off (default), left eye optic disccentered, left eye macula centered, right eye macula centered, and righteye optic disc centered.

An example method allows for manual adjustment of settings for imageacquisition. In this example, the clinician has access to varioussettings that can be adjusted manually, such as PET and focus andbrightness. See FIG. 38 described below.

An example method for adding images includes, once an image is captured,the clinician manually adding the image to a workspace if desired. Onceadded, the image can be reviewed and stored, if desired. In thisexample, up to four images can be added to a workspace for review. Otherconfigurations are possible. See FIG. 37 described below.

An example method for entering advanced settings such settings asvolume, time, date, etc. can be accessed upon entering of a password oraccess code by the clinician. In one method, an access code is needed tochange certain settings, and an advanced settings code is needed tochange other advance settings. Other configurations are possible.

In an example method for selecting network connectivity, a plurality ofWiFi networks are shown, and the clinician can select one for connectionthereto. Upon successful connection, the system 600, 605 can communicatewith the cloud.

In an example method for image inspection, once an image is selected, itis displayed to the clinician for review. The user can discard the imageor move forward with image capture, as desired.

In an example method for discard of an image, a number of discards istallied. If over a threshold amount (e.g., 2, 3, 5, 10, etc.), a warningcan be given that further image acquisition could be uncomfortable forthe patient.

In an example method for returning to a home screen, a home button isprovided on each interface. When selected, the home screen interface isshown, allowing the clinician to make initial selections like patientlist, examination, review, and settings.

If the home button is selected when there are unsaved images, theclinician is first prompted to save or discard the images beforereturning to the home screen. In this example, the method includesdisplaying a prompt with a save button to allow the clinician to savethe images. Once saved, the home screen is displayed.

In an example method for docking the system 600, 605, the system 600,605 is placed in a charging cradle. Upon connection with the cradle, anicon indicating a USB connection is displayed on the dock and/or thesystem 600, 605. If acquisition is complete, the screen is turned offand sleep is instituted without a certain time period (e.g., oneminute). If acquisition is not complete, the clinician is prompted tocomplete acquisition.

In an example method for assuring that all items for an examination havebeen received or overridden, if items are missing, the save button isdisabled. However, the clinician can select the override button and, incertain contexts, allow for saving of data without all required items(e.g., a skipped indication) being present.

In an example method for updating software on the system 600, 605,software can be uploaded from a removable storage medium (e.g., SD card)during boot to update the software on the system 600, 605. In otherexamples, software can be downloaded, such as from the cloud.

In another example for waking the system 600, 605 from sleep, the usercan press the home button to wake the system. Upon wake, a login screencan be presented, requiring the clinician to enter an access code to usethe system 600, 605.

In some examples a method is provided for training purposes. In thisembodiment, training information can be accessed from the home screen.The training can provide user interface information that trains the useron the configuration and use of the system 600, 605.

Referring now to FIGS. 34-35, in some example, the system 700 allows formessaging to the clinician who is capturing the fundus images. Forexample, the cloud system 704 and/or the clinicians working as part ofthe over read service 712 can directly message the clinician capturingthe fundus images regarding such as issues as image quality.

For example, FIG. 34 shows a method 720 that allows the over readservice to message the clinician obtaining the fundus images with thedevice 702. Such messages can be addressed using various methods, suchas device name, device ID (serial number/MAC address), device IPaddress, etc. In this example, one or more messages are provided by theover read service 712 to the cloud system 704. At operation 722, thedevice 702 connects to the cloud system 704 using a known protocol, suchas TCP/IP. At operation 724, a determination is made regarding whetheror not a message is waiting for the device 702. If so, control is passedto operation 726, and a determination is made regarding whether or not aparticular graphical user interface (e.g., a home screen) is beingdisplayed on the device 702. If so, control is passed to operation 728,and a message is presented to the clinician on the graphical userinterface.

At FIG. 35, one example of such a message 729 is shown. The message 729can be displayed so as to get the attention of the clinician operatingthe device 702, such as by popping up, color, sound, etc. The message729 can provide information regarding the quality of the images thathave been captured by the device 702. For example, if the images are notof a sufficient quality for the over read service 712, the over readservice 712 can send a message to the device 702. The clinician C canread the message, as well as information about how to remedy thesituation (e.g., the message could provide information such as “Clean acertain part of the lens, etc.).

In addition to the messaging between the device 702 and the cloud system704 described above, the cloud system 704 can be used to store variousinformation associated with the examination of a given patient. Forexample, as the fundus images are captured, the clinician C can adjustvarious settings associated with the device 702, such as brightness,focus, etc. Once a desired set of settings is identified for aparticular patient, these settings can be stored in the cloud system 704(e.g., in a database) and/or the EMR 706 and associated with thepatient. When the patient returns for a subsequent examination, thedevice 702 can be configured to automatically access the settings forthe device 702 by downloading the settings from the cloud system 704. Inthis manner, the device 702 can be automatically configured according tothose settings for subsequent capture of the patient's fundus images.

Referring now to FIG. 36, an example workflow 730 for automaticallycapturing fundus images using the device 702 is shown. The workflow 730is automatically performed by the device 702 to provide a standardizedfundus examination. The workflow 730 includes a selection stage 732, apre-acquisition stage 734, an acquisition stage 736, and apost-acquisition stage 738.

At the selection stage 732, the clinician C is presented with a menu ofoptions, including an examination icon. The clinician C selects theexamination icon to initiate the workflow 730.

At the pre-acquisition stage 734, the clinician C is presented by thedevice 702 with options to start the workflow 730 or to perform a manualcapture of fundus images (see FIG. 37). The clinician C selects the“Start” button 735 to begin the workflow 730 (or can select the manualcapture icon 737 to manually capture images as described further below.

At the acquisition stage 736, the device 702 automatically captures thedesired fundus images from the patient P. The image capture can includeone or more tones indicating the capture of images, along with automatedquality checks on the images. An example of such a process forautomating the capture of fundus images is described in U.S. patentapplication Ser. No. 15/009,988.

Finally, at the post-acquisition stage 738, the clinician C can reviewthe captured images. The clinician C can perform such actions asdiscarding images and/or adding images, as described further below.

For example, the clinician C can decide to discard one or more of thefundus images. In one example, the clinician C is provided with variousoptions. If one option is selected (e.g., a “Close” icon 742), thedevice 702 returns to the pre-acquisition stage 734. If another optionis selected (e.g., a trash icon 744), the device 702 returns to theacquisition stage 736 to allow for the immediate retake of the fundusimage(s). Other configurations are possible.

In another example, clinician C can add images for the patient P. Inthis example shown in FIG. 37, a user interface includes a control 748that allows the clinician C to add images by returning the device 702 tothe pre-acquisition stage 734. At that point, the device 702 can be usedto capture further fundus images that are associated with the patient P.

In addition, other workflows can be performed by the device 702. Forexample, the workflow 730 can be a default workflow for the device 702,but the device 702 can be configured to perform a modified workflowdepending on which over read service 712 is used. For example, aparticular over read service 712 may be defined requirements for suchparameters as: number of fundus images; type of fundus images (e.g.,left, right, macula entered, optic disc centered, etc.); order ofcapture sequence.

In some examples, the workflow for the device 702 is defined by one ormore scripts. The scripts can be downloaded from the cloud system 704 toallow for the modification of the functionality of the device 702. Aparticular script can be selected by the clinician C to modify theworkflow for the device 702. In addition, the device 702 can beprogrammed to automatically select a script based upon such parametersas clinician C preference, over read service, etc.

In addition to the automated workflow 730, other configurations arepossible. For example, as part of the automated capture of fundusimages, the clinician C can select to manually capture one or morefundus images. Specifically, during the pre-acquisition stage 734 or theacquisition stage 736, the clinician C can select one of the manualcapture icons 737, 758 to have the device 702 capture an image. Otherconfigurations are possible.

Referring now to FIG. 39, in some examples, the pre-acquisition stage734 allows the user to pre-select an eye position and fixation targetbefore taking fundus image(s). In this example, controls 762 areprovided that allow the clinician C to select eye position (e.g., left,right, macula entered, optic disc centered, etc.) before images areeither automatically (by selecting “start”) and/or manually (byselecting “manual”) captured using the device 702.

When manually capturing images, the device 702 is programmed as depictedin FIG. 40 to assist the clinician C with aiming the device 702 forcapturing the fundus image(s). In this example, a circular element 770guides the clinician C with the initial approach of the device 702 tothe eye. A target element 772 thereupon provides the user withadjustment guidance once the device 702 is in the correct proximity tothe patient P's eye. Specifically, when the device 702 is focused in theinner eye, a reflection will appear in the view from the cornea. Theclinician C can thereupon use micro adjustments of the barrel of thecamera to move the reflection into the target element 772. Once inposition, the device 702 will automatically trigger a capture of theimage.

With the reflection positioned in the target, a good image of the funduscan automatically be captured. However, in some scenarios, it can bechallenging for the clinician C to position the reflection within thetarget element 772 to initiate automatic capture of the image. Forexample, the clinician C may not be familiar with the operation of thedevice 702. Or, the fundus can be small (such as in younger patients),which can result in greater difficulty maneuvering the device 702 toposition the reflection within the target element 772.

Referring now to FIGS. 41-45, another example is provided of the device702 programmed to assist the clinician C with aiming the device 702 forcapturing the fundus image(s).

In the example of FIGS. 41-45, a graphical user interface 802 isprovided on the display 602 of the device 702. Aspects of the interface802 are modified over time to assist the clinician C in capturing thefundus images.

In FIG. 41, the interface includes the target element 772 and thereflection 804. This is similar to the example of FIG. 40, in that thetarget element 772 is provided as a target. The clinician C manipulatesthe position of the device 702 to position the reflection 804 within thetarget element 772, at which time the device 702 will automaticallytrigger a capture of the image.

However, if a period of time passes and the clinician C is unable toposition in the reflection 804 into the target element 772, theinterface 802 is modified so that the relative size of the targetelement 772 is increased.

For example, as shown in FIG. 42, if a specified period of time elapsesand the reflection 804 has not been positioned within the target element772, the relative size of the target element 772 is increased from itsoriginal size on the interface 802 of FIG. 41. This allows the clinicianC to more easily position the reflection 804 into the target element772, since the target element 772 is larger on the interface 802.

As shown in FIG. 43, if a further specified period of time elapses andthe reflection 804 has not been positioned into the target element 772,the relative size of the target element 772 is increased further fromits original size on the interface 802 of FIG. 41. This again allows theclinician C to more easily position the reflection 804 into the targetelement 772, since the target element 772 is larger on the interface802.

As shown in FIG. 44, if yet another further specified period of timeelapses and the reflection 804 has not been positioned into the targetelement 772, the relative size of the target element 772 is increasedfurther from its original size on the interface 802 of FIG. 41. Thisagain allows the clinician C to more easily position the reflection 804into the target element 772, since the target element 772 is larger onthe interface 802.

In the example shown, the target element is shaped as a diamond. Inother examples, other shapes can be used. For example, in alternatives,the target element can be a triangle, square, rectangle, or circle.

Referring now to FIG. 45, an example method 900 is shown to modify thegraphical user interface 802 to assist with aiming during capture ofimages.

Initially, at operation 902, the interface including the target (e.g.,target element 772) and reflection are shown. Next, at operation 904, adetermination is made regarding whether or not a specified time haselapsed without capture of the image. If not, the process is ended.

However, if a specified time has elapsed without capture, control ispassed to operation 906, and the requirements for capture are modified(e.g., the target size is increased or other modifications made, asdescribed herein). Control is then passed back to operation 902.

In the example depicted in FIGS. 41-45, the size of the target isincreased in specified intervals. One example of this progression is asfollows:

-   -   FIG. 41—target is original size;    -   FIG. 42—2.5 seconds elapses, then target is increased        approximately 20 percent from the original size;    -   FIG. 43—another 2.5 seconds elapses, then target is increased        approximately 50 percent from the original size; and    -   FIG. 44—another 5 seconds elapses, then target is increased        approximately 100 percent from the original size.

This process can continue further as additional time elapses. Inaddition, other intervals and size increases can be used. For example,the intervals can be evenly spaced (e.g., every 2 seconds, 3 second, 5seconds, 7 seconds, 10 seconds, etc.) or graduated as provided in theexample (e.g., 2 seconds, 4 seconds, 6 seconds, 8 seconds, etc.).

Also, the size modifications of the target can be different. Forexample, the target can grow more quickly or slowly. And, in otherexamples, the shape of the target can also be modified to make it easierfor the clinician C to position the reflection from the cornea withinthe target.

In yet other examples, other aspects of the interface 802 can bemodified in addition to or in place of the size of the target to furtherhelp the clinician C to automatically capture the fundus image(s).

For example, in one alternative, the clinician C can be presented with acontrol that allows the clinician C to request assistance if theclinician C has difficulty. This control can be actuated by theclinician C to manually request that the size of the target beincreased. This can be in place of or in conjunction with the automaticchanges in size described above.

For example, the device 702 has certain requirements regarding theneeded intensity of the reflection from the cornea before an image canbe captured. In one example, the reflection must exhibit a contrast of20 percent or more before the reflection can be used to trigger theautomatic capture of the fundus image(s). If the contrast is less than20 percent relative to the remaining image of the eye on the display 602of the device 702, the clinician C must modify the given conditions(e.g., modify lighting, move the device, etc.) to create a greaterreflection intensity (i.e., greater contrast) before the image can becaptured.

However, it may be difficult for the clinician C to accomplish thegreater contrast. If so, over time, the threshold contrast requirementcan be lowered to allow the clinician C to more easily automaticallycapture the images. For example, after a specified duration, such asthose provided above, the threshold contrast can be decreased to 10percent. A further decrease to 5 percent can be done after anotherelapsed duration. The given values are just examples, and other valuescan be used for the threshold and durations.

The description and illustration of one or more embodiments provided inthis application are not intended to limit or restrict the scope of theinvention as claimed in any way. The embodiments, examples, and detailsprovided in this application are considered sufficient to conveypossession and enable others to make and use the best mode of claimedinvention. The claimed invention should not be construed as beinglimited to any embodiment, example, or detail provided in thisapplication. Regardless whether shown and described in combination orseparately, the various features (both structural and methodological)are intended to be selectively included or omitted to produce anembodiment with a particular set of features. Having been provided withthe description and illustration of the present application, one skilledin the art may envision variations, modifications, and alternateembodiments falling within the spirit of the broader aspects of theclaimed invention and the general inventive concept embodied in thisapplication that do not depart from the broader scope.

What is claimed is:
 1. An apparatus for producing a fundus image, theapparatus comprising: a processor and a memory; an illuminationcomponent including a light source; a camera; and a display, wherein thememory stores instructions that, when executed by the processor, causethe apparatus to: display a target element on the display; display alight source reflection from a cornea of an eye on the display; update aposition of the light source reflection on the display as a caregivermanipulates a position of the apparatus relative to the eye; as timeelapses, modify the display to allow the light source reflection to moreeasily be positioned within the target element; and automaticallyinitiate fundus image capture with the camera when the light sourcereflection is within the target element on the display.
 2. The apparatusof claim 1, wherein the display is modified to change a size of thetarget element or a threshold for the light source reflection.
 3. Theapparatus of claim 2, wherein the light source is an infrared lightsource.
 4. The apparatus of claim 2, wherein the size of the targetelement is increased as the time elapses.
 5. The apparatus of claim 2,wherein the size of the target element is increased multiple times asthe time elapses.
 6. The apparatus of claim 2, wherein the threshold forthe light source reflection is lowered as the time elapses.
 7. Theapparatus of claim 1, wherein the target element is shaped as a diamond.8. The apparatus of claim 1, wherein the target element increases insize in periodic intervals.
 9. The apparatus of claim 8, wherein an areaof the target element increases in size one hundred percent.
 10. Amethod for capturing a fundus image, the method comprising: displaying atarget element; displaying an infrared light source cornea reflectionfrom an eye; updating a position of the infrared light source corneareflection as a position of the apparatus relative to the eye changes;as time elapses, increasing a size of the target element; andautomatically initiating fundus image capture with the camera when theinfrared light source cornea reflection is within the target element onthe display.
 11. The method of claim 10, further comprising increasingthe size of the target element multiple times as the time elapses. 12.The method of claim 10, further comprising decreasing a threshold forthe light source reflection as the time elapses.
 13. The method of claim10, further comprising shaping the target element as a diamond.
 14. Themethod of claim 10, further comprising increasing a size of the targetelement in periodic intervals.
 15. The method of claim 10, furthercomprising increasing an area of the target element in size one hundredpercent.
 16. A method for capturing a fundus image, the methodcomprising: displaying a target element; displaying an infrared lightsource cornea reflection from an eye; updating a position of theinfrared light source cornea reflection as a position of the apparatusrelative to the eye changes; as time elapses, lowering a threshold forthe infrared light source cornea reflection; and automaticallyinitiating fundus image capture with the camera when the infrared lightsource cornea reflection is within the target element on the display.17. The method of claim 16, further comprising increasing a size of thetarget element multiple times as the time elapses.
 18. The method ofclaim 16, further comprising shaping the target element as a diamond.19. The method of claim 16, further comprising increasing a size of thetarget element in periodic intervals.
 20. The method of claim 16,further comprising increasing an area of the target element in size onehundred percent.